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Abstract 

On the set of genotypes <3? = {1, • • • , m} we introduce a binary relation generated 
by Volterra quadratic stochastic operator V on (m — 1) dimensional simplex 5 m_1 
and prove that the operator V be non-ergodic if either there exists a Hamiltonian 
cycle or one of the vertices Mi = (5u, 621, • • • , S m i) of the simplex S"™" 1 is a source 
and restriction of V to the invariant face Fi = {x E S" 1 " 1 : X{ = 0} is non-ergodic. 
In this paper we prove this result for m = 2, 3,4. 
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1 Introduction 

Quadratic stochastic operators were first introduced by Bernstein p]. Such operators 
frequently arises in many models of mathematical genetics, namely, theory of heredity 
PP,[0]-[in]- Consider a biological population, that is a community of organisms closed 
with respect to reproduction. Assume that each individual in this population belongs to 
precisely one species (genotypes) 1, • • • , m. The scale of species is such that the species 
of the parents % and j unambiguously determines the probability of every species k for 
the first generation of direct descendants. Denote this probability, that is to be called the 
heredity coefficient, by Pij t k- It is then obvious that > for all i,j, k and that 

m 

^2pij,k = l(i,j,k = ,m). 
fc=i 
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The state of the population can be described by the tuple (xi,x 2 , ■ ■ ■ ,x m ) of species 
probabilities,that is x^ is the fraction of the species k in the total population. In the 
case of panmixia (random interbreeding) the parent pairs % and j arise for a fixed state 
x = (x\, x%, ■ ■ ■ , x m ) with probability XiXj. Hence the total probability of the species k in 
the first generation of direct descendants is defined by 

m 

^ Pij,kXiXj, {k = 1, • • • , Tfl) 

Let 

in 

S"™ -1 = {x = (xi, X2, ■ ■ • , x m ) G R m : for any i Xi > 0, and x,- L = 1} (1) 

i=i 

be the (m — l)-dimensional simplex. A map V of S m ~ l into itself is called a quadratic 
stochastic operator (q.s.o.) if 

m 

(Vx) k = 22 PihkXiXj (2) 

for any x G 5 1 " 1 " 1 and for all k — 1, • • • , m, where 

m 

a)Pij,k > 0, b)p ijtk = Pji,k for all i, j, k; c)}^^ = 1. 

fc=i 

Assume {x^ G S" 71-1 : n = 0, 1, 2, • • • } is the trajectory of the initial point x G S"™ -1 , 
where x( n+1 ) = V(x.^) for all n = 0, 1, 2, • • • , with = x. 

Definition 1.1 A point a G S™ 1 ' 1 is called a fixed point of a qso if V(a) = a. 
Definition 1.2 A qso V is called regular if for any initial point x G S m ~ l the limit 

lim V n (x) 

n— >oo 

exists. 

Note that the limit point be a fixed point of a qso V. Thus the fixed points of qso 
describe limit or long run behavior the trajectories of any initial point. Limit behavior 
of trajectories and fixed points of qso play important role in many applied problems 
[3], [1], 0)0, [S] • The biological treatment of the regularity of qso is rather clear: in long 
run the distribution of species in next generation coincide with distribution of species in 
previous one, i.e., stable. 

Definition 1.3 A qso V is said to be ergodic if the limit 

1 n— 1 

lim-Vv fc (x) (3) 

fc=0 
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exists for almost all x G S m ~ L with respect to usual Lebesgue measure on S m ~ L . 

On the basis of numerical calculations Ulam conjectured [TT] that the ergodic theorem 
holds for any qso V. In 1977 Zakharevich [12] proved that this conjecture is false in 
general. Later in [2] was established necessary and sufficient condition to be non-ergodic 
transformation for qso defined on S 2 . The biological treatment of non-ergodicity of qso 
V is the following: in long run the behavior of distributions of species is chaotic, i.e., 
unpredictable. 

Note that a regular qso V is ergodic, but in generally from ergodicity does not follow 
regularity. Let 7r be a permutation of a set {1, 2, • • • , m}. Then one can define one-to-one 
transformation : S 771 ' 1 — > S m ~ L as 

T n (x 1 ,X 2 , ■ ■ ■ ,X m ) = (x 7r (i),X 7r ( 2 ), • • ■ ,Xir(m)) (4) 

It is evident that a transformation T~ 1 VT 7T is qso and limit behavior of trajectories of qso 
T^VT^ coincide with limit behavior of trajectories of qso V [5]. Thus for any permutation 
it a qso T~ X VT^ be regular (resp. ergodic) if and only if a qso V is regular (resp. ergodic). 

The following notations will be used throughout this paper [7J. We let OS 171 ' 1 denote 
the boundary of S' m_1 : 

dS m-i = | x e gm-i :Xi = for at least one z G {1, 2, • • • , m}}, 
and also for i — 1, 2, • • • , m : 

Mi = (5u, 6 2 i, • ■ ■ , S m i) be ith vertex of the simplex 5 1 " 1 " 1 , 

Fi be the ith face of S m ~\ where F { = {x e S™- 1 : x t = 0}; 

and interior of S m ~ L be the set intS" 1 ^ 1 = {xG S m ~ L : X1X2 ■ ■ ■ x m > 0}. 

Also denote by co(x°) to-set of limit points of the trajectory {x^ n \ n = 0, 1, 2, ...}. 

2 Extremal Volterra Quadratic Stochastic Operators 
and Tournaments 

Let $ = {1, ■ ■ ■ ,m} be the scale of genotypes and V a quadratic stochastic operator . 
Definition 2.1 The quadratic stochastic operator V is called Volterra, if p^fc = for 
any k £ {i,j}. 

The biological treatment of such operators is rather clear: the offspring repeats one of its 
parents. 

Evidently for any Volterra qso paj = 1 for any % = 1, • • • , m. A Volterra qso V defined on 
S 1 " 1-1 has following form 

m 

(Vx) fe = x\ + 2^2p iktk XiX k , (5) 

where k = 1, • • • , m. 

Proposition 2.1 [3] [7] A qso V is a Volterra if and only if 

m 

(Vx) fc = x k (l + a ki Xi) (6) 
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where A = ( skew-symmetric matrix with a^i = ^Pik,k ~ 1, and \a^\ < 1. Here 

ij G {1,2, • • • ,m}. 

Proposition 2.2 Let V be a Volterra qso. Then for any permutation 7r G S" m the 
transformation T^KT^ is the Volterra qso. 

Proof For qso V (5) and permutation n E S m simple algebra gives 

m 

(T;VT T x) fc = x£ + 2^p I -i ( ^-i Wi7r -i W i 1 i i , (7) 

i^k 

where k — 1, • • • , m. It is evident that the transformation (7) is the Volterra qso. 

In [3] and [I] the theory of Volterra qso was developed using theory of the Lyapunov 
functions and tournaments. Below we will consider a class of so-called extremal qso. 
Definition 2.2 The quadratic stochastic operator V is called extremal Volterra, if 
Pij,k = or 1 for k G {i, j}. 

Let £ be the set of all extremal Volterra qso on 5' m ~ 1 . Since p^ = 1 ~ Pik,k, the total 
number of mixed products x,iXj with i ^ j in (5) is equal to m(m — l)/2. Note that if 

V be an extremal Volterra qso then elements of matrix A take only two values ±1. Thus 

m ( m — 1 ) 

there are 2 2 extremal Volterra qso. 

Proposition 2.3 Let V be an extremal Volterra qso. Then for any permutation 7r G S m 
the transformation T~ l VT 7T is the extremal Volterra qso. 
Proof The proof immediately follows from Proposition 2.2. 

Definition 2.3 Let V\, V% G £ be two extremal Volterra qso. We will say that the qso 
V\ is equivalent to V2 and denote V\ ~ V% if there exists a permutation 71 G S m such that 
VJ, = T^V X T„. 

Since this relation is equivalence one we can part the set £ into equivalence classes. Let 

V be extremal Volterra quadratic stochastic operator. Then on the set $ = {1, • ■ ■ ,m} 
of genotypes one can introduce following a binary relation: if p^- j = 1 we will say that 
the genotype i dominate genotype j and denote i >~ j. Any two genotypes i and j are 
comparable, i.e., i y j or j >~ i. Let the set $ = {1,- •• ,m} is provided with directed 
graph structure where the edge connecting genotypes i and j directed from i to j if i y j. 
That is, it is a directed complete graph in which every pair of vertices is connected by a 
single directed edge. Such graphs is called a tournament. 

Definition 2.4 A cycle of a graph $ is a subset of the edge set of $ that forms a path 
such that the first node of the path corresponds to the last. 

Definition 2.5 A cycle that uses graph vertex of a graph exactly once is called a Hamil- 
tonian cycle. 

A graph containing no cycles of any length is known as an acyclic graph, whereas a graph 
containing at least one cycle is called a cyclic graph. 

Let the indegree of a vertex is the number of edges leading to that vertex, and the outde- 
gree of a vertex is the number of edges leading away from that vertex. A vertex with an 
indegree of is called a source (since one can only leave it) and a vertex with an outdegree 
of is called a sink (since one cannot leave it). 

Proposition 2.4 If for given qso V a vertex Mi = (5u, 621, ■ • • , S m i) of the simplex S" 1 ' 1 
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is a source or sink, then the face Fi = {x G S m 1 : Xj = 0} is invariant subset. 
Proof The proof immediately follows from definition of source and sink. 



3 Ergodicity of Extremal Volterra Operators 

Let $ = {1, • • • , m} be a set of genotypes. In this paper we consider small m with 
m = 2, 3, 4. General case will be subject of next paper. 

Theorem 3.1 An extremal Volterra quadratic stochastic operator V be non-ergodic if 
either there exists a Hamiltonian cycle or one of the vertices M{ = (6u,8zi, ■ ■ ■ ,S m i) of 
the simplex S" 1 " 1 is a source and restriction of V to the invariant face Fi = {x G S*" 1-1 : 
Xi = 0} is non-ergodic. 

Proof For k = 2 we have two extremal Volterra qso 

x[ = x\ + 2axiX 2 , 

(8) 

x' 2 = io + 2(1 — a)xiX2 



where a G {0, 1}. Since we have two genotypes only, the corresponding graph is acyclic 
and simple analysis shows that the operator (8) is the regular. It is evident that the qso 
(8) has two fixed points Mi = (1, 0) and M 2 = (0, 1). For a = (respectively a = 1) any 
trajectory converges to M 2 (respectively Mi). 

Let k = 3. Then we have 8 extremal Volterra operators 

+ 2axiX 2 + 2/3xiX 3 , 

x' 2 = x\ + 2(1 — a)x\X 2 + 27x2X3 (g) 

x' 3 = x\ + 2(1 - (3)xix 3 + 2(1 - 7)x 2 x 3 

where a, /3, 7 G {0, 1}. It is easy to see that there exists Hamiltonian cycle with respective 
binary relation generated by qso with (a, {3,j) = (0,1,0) or (a,/?, 7) = (1,0,1) and for 
remaining 6 cases corresponding graph is acyclic. The qso (9) with (a,(3,j) = (1,0,1) 
has following form: 

x[ = x\ + 2xiX 2 , 

x' 2 = x 2 2 + 2x 2 x 3 (10) 

X'n = X\ + 2XlX3 



This operator was considered by Zakharevich [12] and was proven that it is non-ergodic 
transformation. Second operator with (a, /3, 7) = (0, 1,0) is reduced to (10) by a permu- 
tation of genotypes. For (a, 0, 7) = (0, 0, 0) the corresponding qso has following form: 



Jj — 1 ; 



x' 2 = x 2 2 + 2x l x 2 (11) 
x 3 = x| + 2x1X3 + 2x2X3. 
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and all others qso with acyclic graph are reduced to (11) by some permutation of geno- 
types. Note that there exist exactly 6 permutations and respectively 6 qso with acyclic 
graph. Simple analysis shows that qso (11) is recurrent. One can rewrite qso (11) as 
following: 

x 1 = x 1 , 

x' 2 = xl + 2x x x 2 (12) 
x' 3 = x 3 (2 - x 3 ). 

It is evident that one-dimensional transformation tp(x) — x(2 — x) has two fixed points 
x* = and x** = 1 with <//(0) > 1 and tp'(l) < 1, such that x* is repelling (resp. x** 
attracting ) fixed point. Since <p(x) is increasing function on segment [0, 1] and x** is fixed 
point, then lim^^ ^ n \x) = 1, where ip( n \x) = </?(</?( n_1 )(x)), therefore any trajectory of 
qso (12) converges to vertex (0,0, 1), except fixed point (1,0,0). 
For k = 4 the set £ consists of 64 extremal Volterra operators 

x[ — x\ + 1ax\x<i + 2/3xiX3 + 27x1X4, 

x' 2 = x 2 + 2(1 — a)xix 2 + 25x 2 x 3 + 2ex 2 x 4 

x' 3 = x 2 3 + 2(1 - (3)x lX3 + 2(1 - 5)x 2 x 3 + 2\x 3 x A ^ 
x' A = x\ + 2(1 - i)x x x A + 2(1 - e)x 2 x A + 2(1 - X)x 3 x 4 



where a,j3, 7, 5, e,X G {0,1}. Simple but tedious algebra gives that the set 8 is parted 
into 4 equivalent classes. First class consists of 24 extremal Volterra operators such that 
there exists Hamiltonian cycle. Second class consists of 8 extremal Volterra operators 
such that for corresponding graph one vertex is a sink and other 3 vertices forms three 
cycle. Third class consists of 8 extremal Volterra operators such that for corresponding 
graph one vertex is a source and other 3 vertices forms three cycle. Finally, fourth class 
consists of 24 extremal Volterra operators such that the corresponding graphs are acyclic. 
Note that for k = 4 there exist exactly 24 permutations. 

3.1 First Equivalent Class 

lia = (3 = 5 = e = \ = l and 7 = then there exists the Hamiltonian cycle 1 >- 2 y 3 >- 
4^1 and corresponding qso (13) has following form 

x[ — x\ + 2a;ia;2 + 2x1X3, 
x' 2 = x\ + 2x2X3 + 2x2X4 
X3 = x\ + 2x 3 x 4 

The equivalent class containing this operator consist of 24 extremal Volterra qso and 
for each of them corresponding tournament contains the Hamiltonian cycle. It is easy 
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to see that the set of fixed points contains following points Mi(l, 0,0,0), M 2 (0, 1,0,0), 
M 3 (0, 0, 1, 0), M 4 (0, 0, 0, 1), M 5 (l/3, 0, 1/3, 1/3), M 6 (l/3, 1/3, 0, 1/3). Introducing new vari- 
ables y% = xi, yi = x 2 + x 3 , = x±, one can see that 

V\ = X li 2/2 = X 2 + ^3' 2/ 4 = X 4> 

and qso (14) reduced to following 

Vi = vi + 22/12/2, 

2/2 = 2/1 + 2 2/22/3 (15) 
2/3 = 2/1 + 2 2/i2/3 



Note that this qso (114"]) has following invariant subset {rM 5 + (1 — r)M 6 , r G [0, 1]} and 
trajectory of (fl4"|) with initial point from these invariant subset converges to fixed point 
M 6 . Since Lebesgue measure of invariant subset is equal to 0, this qso is non-ergodic. 
Thus we have proved following Proposition. 

Proposition 3.1 Any qso from first class is non-ergodic operator. 



3.2 Second Equivalent Class 



e = then the vertex Mi = (1, 0, 0, 0) is the sink, other 
2 y 3 y 4 y 2 and corresponding qso (13) has following 



Ifa = /3 = 7 = <5 = A = 1 and 
three vertices forms three cycle 
form 



x\ + 2xiX 2 + 2x1X3 + 2xiX 4 , 
x\ + 2x 2 x 3 
x\ + 2x3X4 
X4 + 2x2X4. 



(16) 



The equivalent class containing this operator consist of 8 extremal Volterra qso and for 
each of them corresponding tournament contains three cycle. It is easy to see that the 
set of fixed points consists of following points Mi(l, 0, 0, 0), M 2 (0, 1, 0, 0), M 3 (0, 0, 1, 0), 
M 4 (0, 0, 0, 1), M5(0, 1/3, 1/3, 1/3). First equality in (16) one can rewrite as x'i = xi(2 — xi) 
similar (12). As proved above, for any initial x° G dS 3 its trajectory converges to vertex 
Mi = (1,0,0,0). It is evident that the face Fi = {x G S 3 : Xi = 0} is invariant subset 
with Lebesgue measure and restriction of qso (16) to F% is reduced to Zakharevich's 
example (10). Thus we have proved following Proposition 

Proposition 3.2 Extremal Volterra qso V belonging to the second class is ergodic. 



3.3 Third Equivalent Class 

lia = (3 = 6 = e = l and 7 = A = then the vertex M 3 = (0, 0, 1, 0) is a source , other 
three vertices forms three cycle 1 >- 2 y 4 >- 1 and corresponding qso (13) has following 
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form 

x[ = x\ + 1X\X<i + 2xix 3 , 

x' 2 = x\ + 2x 2 X 3 + 2X2^4 



x 3 — x 3 



4 = x\ + 2x x x 4 + 2x 3 x 4 



The equivalent class containing this operator consist of also 8 extremal Volterra qso and 
for each of them corresponding tournament contains three cycle. It is easy to see that the 
set of fixed points consists of following points Mi(l, 0,0,0), M 2 (0, 1,0,0), M 3 (0, 0,1,0), 
M 4 (0,0,0,l),M 5 (l/3, 1/3,0,1/3). It is evident that lim x 3 n) = 0. If for initial x° E S 3 

n— >oo 

we have x\ ^ then u(x°) C F 3 = {x e S 3 : x 3 = 0}. It is evident that the face 
F 3 = {x £ S" 1 ^ 1 : x 3 = 0} is invariant subset and restriction of qso (17) on F 3 is reduced 
to Zakharevich's example (10). Thus we have proved following Proposition 

Proposition 3.3 If extremal Volterra qso V belongs to the third class, then it is 
non-ergodic transformation. 



3.4 Fourth Equivalent Class 

Ua = f3 = r y = 5 = e = \ = l then corresponding graph is acyclic and corresponding qso 
(13) has following form 

x[ — x\ + 1X\X2 + 2xiX 3 + 2X1X4, 

x' 2 = x\ + 2x2X3 + 2x2X4 

2 (18) 
+ 2x 3 x 4 

dj 4 — X4 



The equivalent class containing this operator consist of 24 extremal Volterra qso and for 
each of them corresponding tournament is acyclic. It is easy to see that the set of fixed 
points consists of following points Mi(l, 0, 0, 0), M 2 (0, 1, 0, 0), M 3 (0, 0, 1, 0), M 4 (0, 0, 0, 1) 
and the sets F 1 and r 34 = {x e S 3 : X\ = x 2 = 0} are invariant. First equality (18) one 
can rewrite as x[ — Xi(2 — Xi) and using properties of the function (p(x) = x(2 — x) we 
have 

1) If x\ 7^ then any trajectory of qso (18) converges to the vertex Mi = (1, 0, 0, 0); 

2) if an initial point belongs to set F 1 \ T 34 , i.e., x? = 0, then the operator (18) is reduced 
to qso (11), therefore any trajectory converges to the vertex M2 = (0,1, 0, 0); 

3) if x G r 34 \ M 4 then the trajectory converges to third vertex M 3 = (0, 0, 1, 0). Thus we 
have the following Proposition. 

Proposition 3.4 Any extremal Volterra qso V belonging to the fourth class is regular 
transformation. 

Collecting together all four Propositions we complete the proof of Theorem 3.1 
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4 Conclusion 



Above we consider extremal Volterra qso on the set Q with |Q| = m where we limit ourself 
by m = 2, 3, 4. To prove this result for arbitrary m we need to apply more deep properties 
of the tournament and it cycles. 
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